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Quality has become a primary focus in the afterschool 
field—and with good reason. Not only does the focus 
on quality convey a broad commitment to doing good 
work in our afterschool programming, but it also 
requires the afterschool field to make informed choices 
about what actually occurs in afterschool programs. 

While research on program effectiveness offers ample 
evidence that afterschool programs can benefit young 
people in a variety of ways, this same body of research 
demonstrates that not all programs are equally effec¬ 
tive (Granger, 2008). Some programs show positive 
results in many or all major outcome categories. 
Other programs are associated with positive results 
for some but not all outcomes; still others show no 
positive outcomes at all. This variability in effective¬ 
ness highlights the need for a better understanding of 
how successful afterschool programs actually work, 
while keeping in mind the ever-present question of 
how to define success. 
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The focus on quality comes at a pivotal time for the 
afterschool field, as prominent voices debate the useful¬ 
ness of allowing particular outcomes to guide program¬ 
ming. Robert Halpern (2006) has called attention to the 
problems associated with programming that is solely 
outcome-driven. He argues that the afterschool field has 
tended to focus on a narrow set of outcomes, usually 
academic, without fully exploring the range and com¬ 
plexity of relevant developmental tasks. Halpern goes on 
to suggest that we reformulate our expectations of after¬ 
school programs to take into account the range of pur¬ 
poses they serve. 

Focusing on afterschool program quality provides 
the opportunity for just such a reformulation. This arti¬ 
cle will show how the vibrant dis¬ 
cussions of afterschool program 
quality, which encompass both the¬ 
ory and empirical research, increas¬ 
ingly offer policymakers, funders, 
evaluators, and program providers a 
sound basis for making decisions 
about afterschool programming. 

The efforts of the afterschool 
field to define program quality are 
generating an increasingly robust 
literature that highlights the value 
of broadening our focus beyond 
outcome-driven approaches to 
include quality-driven approaches. 

Quality-driven approaches focus on 
day-to-day program processes as they relate both to 
holistic program goals and to individual outcomes. 
Because program quality formulations are increasingly 
informed by both theory and empirical evidence from 
evaluation research, a quality-driven approach remains 
empirically based but is liberated from an exclusive 
focus on outcomes. As a point of convergence between 
theory and results-based or outcome-only approaches, 
the field’s emerging focus on program quality offers a 
new and compelling guide for efforts to manage and 
improve afterschool programming. 

This review article examines formulations of after¬ 
school program quality emerging from outcome 
research, expert and professional consensus, and 
“process” research on how successful afterschool pro¬ 
grams operate. After briefly summarizing current litera¬ 
ture on the effectiveness of afterschool programs, we 
review empirical evidence regarding the characteristics 
of effective programs, that is, what effective afterschool 
programs do to produce positive outcomes. Next, we 


provide an overview of how program quality has been 
conceptualized by afterschool researchers. We conclude 
by discussing the field’s emerging consensus regarding 
quality afterschool programming, highlighting key 
considerations as the afterschool field undertakes 
efforts to achieve both program quality and positive 
outcomes for youth. 

Program Effectiveness: Do Afterschool Pro¬ 
grams Make a Difference? 

Not surprisingly, research on afterschool program effec¬ 
tiveness has traditionally focused on outcomes. Studies 
of afterschool program effectiveness document a variety 
of benefits associated with program participation. 

Results of the many evaluations of 
afterschool programs have been 
summarized in several recently 
published qualitative reviews (Bod- 
illy & Beckett, 2005; Little, Wimer, 
& Weiss, 2008). The field has also 
seen the publication of several 
quantitative reviews (Durlak & 
Weissberg, 2007; Lauer et al., 2006; 
Zief & Lauver, 2006) that use the 
technique of meta-analysis to iden¬ 
tify trends in the results of after¬ 
school program evaluations. 

Considered collectively, the 
qualitative and quantitative reviews 
of effectiveness research provide a 
sense of the outcomes afterschool programs are gener¬ 
ally expected to achieve. These reviews also document 
significant variability in programs’ ability to achieve 
these outcomes. Not all afterschool programs are equally 
effective in producing positive youth outcomes. Thus, 
one way to define program quality is to look at the fac¬ 
tors identified in the evaluation research as characteriz¬ 
ing effective programs, defining these as programs that 
produced positive youth outcomes. To the extent that 
reviews of effectiveness research capture variability in 
program practices related to positive youth outcomes, 
these reviews collectively contribute to the field’s efforts 
to define program quality. 

Qualitative Reviews 

Little and colleagues (2008), of the Harvard Family 
Research Project (HFRP), reviewed afterschool evalua¬ 
tion literature and found evidence that programs can 
have beneficial effects on academic performance and 
social and emotional development, as well as preventing 
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problem behaviors and promoting 
health and wellness. Despite this 
list of potential benefits associated 
with participation in afterschool 
programs, Little and colleagues 
concluded that the available 
research indicated that not all 
programs produce these out¬ 
comes. They associate programs’ 
varying levels of success with dif¬ 
ferences in participants’ access to 
and sustained participation in 
programs and with the degree to 
which programs partner with 
families, schools, and other com¬ 
munity contacts. They also associ¬ 
ate the variability in outcome with 
specific program factors they con¬ 
sider to be issues of program qual¬ 
ity. Such factors include the level 
of supervision and structure the 
program provides, the quality of 
staff training, and the degree to 
which program activities are care¬ 
fully matched with the program’s 
specific goals and objectives. 

Bodilly and Beckett (2005) 
found that effective afterschool 
programs achieved positive out¬ 
comes in four categories: provi¬ 
sion of school-age childcare, 
academic attitudes and achieve¬ 
ment, social and health behaviors, 
and social interactions. Bodilly 
and Beckett focused their review 
on only the most methodologi¬ 
cally rigorous evaluations. Using 
these stringent criteria, they 
found that afterschool programs 
were associated with only modest 
benefits. Bodilly and Beckett make 
a compelling case for the importance of taking into 
account factors such as who participates in the after¬ 
school program (age and other personal characteris¬ 
tics), length of time in the program, frequency of 
attendance, program content (specific activities, 
teaching strategies), and level of methodological rigor 
in the evaluation plan. Once again, these dimensions 
of variability are precisely the sort of factors that con¬ 
stitute program quality. 


Quantitative Reviews 

To date, there are three published 
meta-analyses of afterschool pro¬ 
gram evaluations (Durlak & Weiss- 
berg, 2007; Lauer et al., 2006; Zief 
& Lauver, 2006). Meta-analysis is a 
quantitative analysis strategy that 
enables researchers to pool the 
results of many different studies by 
establishing a common metric. 
Consistent with the qualitative 
reviews discussed above, the meta- 
analytic reviews indicated mixed 
results when it comes to afterschool 
program effectiveness. 

The meta-analysis performed 
by Zief and Lauver (2006) yielded 
no evidence that afterschool pro¬ 
grams were effective. Among the 
categories of outcomes they exam¬ 
ined were youth behavioral 
changes, social and emotional 
development, and academic perfor¬ 
mance. Zief and Lauver used very 
strict inclusion criteria, looking 
only at studies that included the 
strongest research designs. They 
therefore included just five studies 
in their meta-analysis. Including so 
few evaluation studies may have 
limited the reliability of their find¬ 
ings. Despite finding that after¬ 
school programs did not 
significantly affect outcomes, Zief 
and Lauver emphasize the impor¬ 
tance of maintaining stringent 
inclusion standards in future meta¬ 
analyses, so that future reviews are 
based on the most rigorous studies 
and so that future evaluations use 
more rigorous experimental 
designs. When it comes to program quality, Zief and 
Lauver’s findings attest to the importance of including 
evaluation design as a dimension of quality. 

The meta-analysis by Lauer and colleagues (2006) 
focused on the effects of out-of-school youth programs 
that were specifically designed to affect academic out¬ 
comes. The findings indicate that, for the most part, pro¬ 
grams focused on academics can produce significant 
benefits in reading and math achievement among the 
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youth they serve. Lauer and colleagues also examined a 
number of factors that might account for variability in the 
degree to which programs produce positive outcomes. 
These factors included students’ grade level, program 
focus, program duration, grouping structure (such as 
individual versus group tutoring), and methodological 
quality of the evaluation study. The meta-analytic find¬ 
ings indicated that several of these characteristics were 
significantly related to variability in both reading and 
math outcomes. However, the effects of these factors 
were not consistent across outcomes. That is, a program 
characteristic that was associated with positive reading 
outcomes was not necessarily related to positive math 
outcomes. For this reason, this meta-analysis indicates 
that afterschool programs can yield benefits in reading 
and math, but it provides little insight into the specific 
factors associated with broad program effectiveness. 
These findings attest to the complex¬ 
ity involved in determining which 
programs work under which circum¬ 
stances for which students. 

The widely referenced meta¬ 
analysis by Durlak and Weissberg 
(2007) focused on afterschool pro¬ 
gramming that was designed to 
improve social skills. Their analyses 
indicate that such afterschool programs can improve 
young people’s academic performance as well as their 
personal adjustment and social skills. Much as Lauer 
and colleagues (2006) did, Durlak and Weissberg 
(2007) considered what factors might be associated with 
positive outcomes. They examined parents’ level of 
involvement in their children’s education; students’ 
grade level; and whether programs included an acade¬ 
mic component that specifically targeted improvement 
in grades, achievement test scores, school attendance, or 
homework completion. In general, their meta-analytic 
findings indicated that none of these factors was consis¬ 
tently associated with positive outcomes. The presence 
of an academic component did emerge as a significant 
predictor, but only on a single outcome—achievement 
test scores. That is, programs that included academic 
programming were more likely to affect achievement 
scores but not other academic outcomes. 

Durlak and Weissberg (2007) identified four char¬ 
acteristics of quality social skills programming that 
were consistently associated with positive outcomes in 
all three outcome domains: personal adjustment, 
social skills, and academic performance. These four 
criteria were: 


• Sequenced set of activities to achieve skill objectives 

• Active forms of learning 

• Focus of at least one program component on devel¬ 
oping personal or social skills 

• Explicit targeting of specific academic, personal, or 
social skills 

These four “SAFE” dimensions were used to sort pro¬ 
grams into two groups. One group consisted of after¬ 
school programs in which all four SAFE criteria were 
present. The other group consisted of programs in 
which only some or none of the SAFE criteria were pre¬ 
sent. Durlak and Weissberg then tested their hypothesis 
by comparing the effectiveness of the group of SAFE pro¬ 
grams to that of the group of non-SAFE programs. Their 
results indicated that SAFE programs that met all four 
criteria showed significant positive effects in seven of the 
eight total outcome domains: child 
self-perceptions, school bonding, 
positive social behaviors, problem 
behaviors, drug use, achievement 
tests, and school grades. Programs 
in the non-SAFE group did not 
show positive results in any out¬ 
come domain. 

These qualitative and quanti¬ 
tative reviews of afterschool program effectiveness 
research indicate that programs can benefit young peo¬ 
ple in terms of their social and emotional functioning, 
academics, health behaviors, and basic safety. As the 
body of program evaluation research grows, what 
emerges as the most striking (and yet commonsensical) 
finding is that program effectiveness varies considerably, 
depending on both the context of the program and on 
what actually occurs in the program. When we focus on 
understanding the dimensions of variability, rather than 
on making sweeping generalizations about all after¬ 
school programs, this effectiveness research makes an 
important contribution to the empirical foundation of 
the program quality movement. 

How or Why Afterschool 
Programs Work: Defining 
Program Quality 

Early program evaluation research provided a snapshot 
of the kinds of factors that contribute to program vari¬ 
ability. However, these evaluation studies were generally 
limited in their ability to answer the question, “Why do 
programs work?” That is, “Which program practices are 
most important in yielding good outcomes?” Until 
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recently, opinions about what constitutes a high-quality 
afterschool program had outpaced available research. 
However, afterschool researchers and evaluators are now 
paying considerable attention to identifying the critical 
elements of successful afterschool programs. 

Varying Perspectives 

Formulations of what constitutes quality in afterschool 
programs have relied on a variety of different sources of 
information, generally including a combination of expert 
opinion and existing empirical research. Expert opinion 
encompasses professionals’ firsthand experiences in plan¬ 
ning and administering afterschool programs, expertise 
in service delivery, and knowledge of adolescent and pos¬ 
itive youth development and the broader field of com¬ 
munity youth development. Frequently, research from 
related fields—such as school-age childcare, youth devel¬ 
opment, and in-school educational 
practice—has been applied to defi¬ 
nitions of quality in afterschool pro¬ 
grams (Little, 2007). This was due 
in part to the limited amount of 
research that had focused specifi¬ 
cally on defining and measuring 
afterschool program quality. 

Afterschool researchers have 
repeatedly noted that the field needs 
more and better empirical evidence 
related to program quality (Bodilly 
& Beckett, 2005; Granger, Durlak, 

Yohalem, & Reisner, 2007; Little, 

2007; Metz, Goldsmith, & Arbreton, 2008; Shernoff & 
Vandell, 2008). The empirical evidence that is available 
consists mostly of descriptive data from correlational 
studies. Correlational studies help to document the co¬ 
occurrence of certain program characteristics with desir¬ 
able outcomes, but these studies cannot explain causal 
links. That is, they cannot determine what program prac¬ 
tices caused improvements in participants’ academic per¬ 
formance or social and emotional development. 

Process measures are another important element of 
more rigorous investigations of program quality. A focus 
on process data, as opposed to an exclusive focus on 
outcome data, provides further insight into the determi¬ 
nants of program quality. Process evaluations generally 
address questions such as, “Who is being served?” and 
“What actually happened in the program?” (Sabatelli, 
Anderson, & LaMotte, 2005). Bodilly and Beckett 
(2005) use the term “implementation analysis” to refer 
to this sort of process evaluation. In afterschool research, 


the process domain encompasses characteristics of par¬ 
ticipants, the broader program context, and what actu¬ 
ally takes place in the program. When combined with 
outcome measures, process data allow researchers to 
document which students, in which programs, under 
which circumstances, have the best outcomes. When 
process-outcome studies include a control condition, 
they provide researchers with information about causal 
relationships between program participation and out¬ 
come. Such information is crucial for determining what 
constitutes a high-quality program. As the field accu¬ 
mulates more empirical data of this sort, it will be able 
to further refine its definition of program quality. 

At this stage of the field’s development, however, 
most formulations of afterschool program quality are 
based on a combination of expert opinion, research from 
adjacent areas of inquiry, correlational studies, and, occa¬ 
sionally, a few more rigorous studies. 
The result has been a number of dif¬ 
ferent definitions of program quality 
that vary in their level of specificity 
(Granger et al., 2007). Many of these 
definitions take the form of concep¬ 
tual models based on afterschool 
researchers’ efforts to integrate 
results from a variety of outcome 
studies into a coherent account of 
optimal program functioning. Other 
definitions of program quality 
emerge “from the field,” that is, from 
the work of practitioners and evalu¬ 
ators who are using observational measures to document 
and evaluate what occurs in afterschool programs. The 
next two sections review definitions of program quality in 
each of these categories. 

Frameworks from Outcome Research 

The afterschool literature reflects a growing number of 
quality frameworks, generally based on a youth devel¬ 
opment perspective, that derive from existing outcome 
research. All of these frameworks offer recommenda¬ 
tions regarding what domains of afterschool program¬ 
ming are most important for achieving positive youth 
outcomes. Some quality frameworks also propose a par¬ 
ticular interrelationship among program domains, 
resulting in a sort of causal theory about how program 
quality affects youth outcomes. The six outcome- 
research-based quality formulations reviewed next have 
clear common themes, as well as a few differences, 
related to afterschool program quality. 
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Metz, Goldsmith, and Arbreton (2008) synthesize 
evidence related to afterschool program outcomes, giv¬ 
ing special attention to the developmental needs of pre- 
teens. Based on afterschool outcome research, Metz and 
colleagues define program quality in terms of six dimen¬ 
sions, some with more research support than others: 

• Focused and intentional strategy 

• Exposure (duration, intensity, and breadth) 

• Supportive relationships 

• Family engagement 

• Cultural competence 

• Continuous program improvement 

Unlike most of the other definitions of quality reviewed 
below, Metz and colleagues not only present these six 
dimensions but also propose a particular relationship 
among them, suggesting that focused, intentional 
programming and continuous program improvement 
are essential for the achievement of quality in the other 
four areas. 

Metz and colleagues’ (2008) formulation also is 
unique in its inclusion of cultural competence as a 
dimension of afterschool program quality. They argue 
that programs that promote cultural competence are 
more likely to have youth who feel psychologically safe, 
actively engaged, and committed to fostering commu¬ 
nity partnerships. Other afterschool professionals have 
called for offering programming that enhances youth’s 
cultural competence, given the diversity of children and 
youth being served in afterschool programs and the 
developmental importance of culture as a dimension of 
identity (The Colorado Trust Afterschool Initiative, 
2002; Kennedy, Bronte-Tinkew, & Matthew, 2007; Oak¬ 
land Afterschool Coordinating Team, 2002; Olsen, 
Bhattacharya, & Scharf, 2006). Metz and colleagues 
suggest that cultural competence is a key dimension of 
program quality that should be infused into all aspects 
of program functioning. 

Researchers at the Forum for Youth Investment 
(Pittman, Smith, & Finn, 2008; Smith & Van Egeren, 
2008) have developed a framework for afterschool pro¬ 
gram quality that emphasizes measuring quality at the 
point of service—where programs come into contact 
with youth and affect what participants actually experi¬ 
ence. This framework is based on program evaluation 
research and developmental theories of motivation and 
learning. These researchers propose that a high-quality 
program is comprised of four hierarchically related 
domains: a safe environment, a supportive environment, 
social interaction, and engagement. According to this 


perspective, providing a safe, supportive environment 
establishes a foundation that fosters effective staff-youth- 
peer interactions. These interactions, in turn, promote 
youth engagement—an element of programming that 
has been shown to correlate with positive youth out¬ 
comes (Smith & Van Egeren, 2008). 

Like the two groups of researchers discussed above, 
Little and colleagues (2008) conclude that the most 
effective programs are those that ensure access to and 
sustained participation in the afterschool program. This 
conclusion is consistent with many studies indicating 
that positive outcomes are more likely when youth par¬ 
ticipate in a program more frequently and over a longer 
period of time (Shernoff & Vandell, 2008). Little and 
colleagues also conclude that effective programs have 
well-established connections with families, schools, and 
other key contexts for youth. Considerable research has 
supported the idea that the greater the number of sup¬ 
portive environments available to youth and the greater 
the consistency across settings in reinforcing positive 
attitudes and behavior, the greater the likelihood that 
youth will show gains in academic achievement, social 
skills, and emotional development (Benson, 2002; 
Chaskin & Baker, 2006; Eccles & Gootman, 2002; 
HFRP, 2004; Shernoff & Vandell, 2008). 

Little and colleagues (2008) also assert that high- 
quality programs also offer appropriate supervision and 
structure for participants, a well-prepared staff, and 
intentional programming. They cite research that relates 
negative youth outcomes to lack of supervision after 
school. The need for a well-prepared staff is supported 
by research that has consistently linked the quality of 
youth-staff relationships both to outcomes and to the 
level of youth engagement in the program. Finally, the 
study refers to the work of Durlak and Weissberg (2007) 
as support for the importance of intentional program¬ 
ming, defined as having clear programmatic goals and 
strong preparation of staff to execute goals. 

Beckett, Flawken, and Jacknowitz (2001) con¬ 
ducted a systematic review of afterschool research, using 
meta-analytic techniques to develop a list of program 
practices associated with positive outcomes. The 
researchers paid careful attention to both the quality and 
quantity of the available evidence supporting each prac¬ 
tice. To be included in their list, a program practice had 
to be mentioned in at least three publications. Each 
practice was given a score indicating the level of research 
support for that practice. The researchers produced a 
list of 18 practices that comprised their formulation of 
program quality. 
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Staff characteristics: 

• Training 

• Education 

• Compensation 

Community contacts: 

• Involvement of families 

• Use of volunteers 

• Partnerships with community-based organizations 

Program characteristics: 

• Variety of activities* 

• Flexibility of programming* 

• Emotional climate* 

• Child-to-staff ratio 

• Total enrollment 

• Mixing of age groups 

• Age-appropriate activities 

• Space availability 

• Continuity and complementarity with day school pro 
grams 

• Clear goals and evaluation of program 

• Materials 

• Attention to health and safety 

Three program characteristics, 
marked with an asterisk above, 
were scored as having strong empir¬ 
ical support: variety of activities, 
flexibility of programming, and 
emotional climate. Most of the 
other practices were found to have 
moderate support. 

More recently, Bodilly and Beck¬ 
ett (2005) reviewed available research 
and theory related to afterschool pro¬ 
gram quality. They examined litera¬ 
ture on youth development, 
school-age care, and quality in edu¬ 
cational settings to develop a list of 
nine afterschool program quality domains: 

• A clear mission 

• High expectations and positive social norms 

• Safe and healthy environment 

• Supportive emotional climate 

• Small total enrollment 

• Stable and well-trained staff 

• Appropriate content and pedagogy (relative to chil¬ 
drens needs and to program’s mission) that provides a 
variety of activities and opportunities for engagement 


• Integrated family and community partnerships 

• Frequent assessment 

Each of the nine quality domains were endorsed by 
at least two of the three literature bases the researchers 
examined. Bodilly and Beckett describe their list as a set 
of program components that are “likely, although not 
proven, to produce effective OST [out-of-school time] 
programming” (p. 73-74). 

Durlak and Weissberg’s (2007) meta-analysis is fre¬ 
quently cited in the reviews discussed above. As mentioned 
previously, Durlak and Weissberg’s four SAFE criteria are: 

• Sequenced set of activities to achieve skill objectives 

• Active forms of learning 

• Focus of at least one program component on devel¬ 
oping personal or social skills 

• Explicit targeting of specific academic, personal, or 
social skills 

Durlak and Weissberg’s findings not only highlight 
the potential importance of these four program char¬ 
acteristics, but also suggest that program quality may 
be holistic. For example, academic benefits may be 
best achieved when a program offers a variety of high- 
quality components, rather than focusing solely on 
strong academic programming. 
Durlak and Weissberg’s findings 
support the conclusion that high- 
quality afterschool programming 
affects a variety of desirable out¬ 
comes, including academic achieve¬ 
ment and social development. 

These six quality frameworks, 
based on outcome research and 
youth development theory, begin to 
explain variability in program effec¬ 
tiveness and move the afterschool 
field closer to a comprehensive defin¬ 
ition of program quality Although the 
field has yet to accumulate a substan¬ 
tial body of empirical evidence concerning the linkages 
between program processes and positive outcomes, these 
recent efforts are an important step forward. 

Observations from Process Measures 

Definitions of program quality are also emerging “from 
the field,” where practitioners and evaluators are formu¬ 
lating quality standards as they use process measures to 
document what is occurring in afterschool programs. 
These observational measures are initially derived from 
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the researchers’ values and assumptions as to what con¬ 
stitutes quality programming. The instruments are used 
by observers, who visit the afterschool programs under 
study and record the extent to which each quality indi¬ 
cator is present in the program. Several groups of after¬ 
school researchers (e.g., HFRP, 2006; Yohalem & 
Wilson-Ahlstrom, 2009) have examined the commonal¬ 
ities among these observational 
instruments and distilled a list of 
core features assessed in these mea¬ 
sures. 

In their compendium, Yohalem 
and Wilson-Ahlstrom (2009) 
selected instruments that were 
“research-based,” that is, “informed 
by relevant child/youth develop¬ 
ment literature” (p. 8). In their 
review of nine instruments, they 
found that six dimensions of qual¬ 
ity were present in all nine observational measures: 

• Focus on staff-student relationships 

• Safe and supportive program environment 

• Active, sustained engagement of youth in program 
activities 

• Encouragement of pro-social behaviors and norms 

• Opportunities for youth to develop specific, targeted 
skills 

• Predictable program structure and routine 

Other domains present in many but not all measures 
included linkages between the program and the com¬ 
munity, quality staffing, and a focus on youth leadership. 

In 2006, researchers at the Flarvard Family Research 
Project (F1FRP) conducted a similar scan of afterschool 
program quality assessment tools. However, they also 
included local, state, and national statements of quality 
standards. Their search resulted in 42 separate articula¬ 
tions of program quality, which they then distilled into 
a list of 15 recurring areas of assessment: 

• Programming, activities, and opportunities 

• Human relationships 

• Positive youth development 

• Family, school, and community involvement 

• Staffing 

• Physical space and environment 

• Program administration and management 

• Safety, health, and nutrition 

• Program planning and structure 

• Assessment, evaluation, and accountability 

• Organizational capacity 


• Supervision and behavior management 

• Sustainability 

• Equal access 

• Fiscal management 

This list of fifteen areas of assessment overlaps sub¬ 
stantially with the domains reflected in Yohalem and 
Wilson-Ahlstrom’s (2009) com¬ 
pendium, as well as with the six 
frameworks based on reviews of 
evaluation research. Considered 
collectively, these eight formula¬ 
tions of afterschool program quality 
indicate that the field is indeed con¬ 
verging on a common vision of the 
essential characteristics of high- 
quality afterschool programs. These 
definitions of program quality 
direct our attention to the domains 
of program functioning that appear to be most impor¬ 
tant for effecting positive outcomes. They also highlight 
the program practices and characteristics, within partic¬ 
ular domains, that are key factors in the success of high- 
quality programs. Finally, several of these definitions of 
program quality contribute to a theory of how various 
dimensions of afterschool program quality are causally 
linked to positive youth outcomes. 

Developing a Unified Picture of 
Program Quality 

The eight quality frameworks reviewed above share clear 
commonalities in the domains of program functioning 
considered essential to program quality. All of them 
address the paramount importance of programs’ provid¬ 
ing youth with safe, supportive relationships and a positive 
emotional climate. Under this broad heading, nearly all of 
the quality frameworks highlight staff contributions to 
establishing a positive emotional climate. Some frame¬ 
works cite structural factors—such as staff being well 
trained, having small staff-to-participant ratios, and hav¬ 
ing adequate staff compensation—as supporting staff in 
establishing strong relationships with young people. 
Some quality frameworks also define positive emotional 
climate in terms of the types of processes or human inter¬ 
actions that occur in the program. 

The various quality frameworks also emphasize the 
importance of afterschool programs’ offering focused, 
intentional programming. Some frameworks emphasize 
program management that is focused and intentional, 
while others stress the importance of focused and inten- 


All of [the quality 
frameworks] address the 
paramount importance of 
programs' providing youth 
with safe, supportive 
relationships and a 
positive emotional climate. 


8 Afterschool Matters 


Fall 2009 



tional activities. Such activities might be chosen to meet 
particular programmatic goals: targeting a specific set of 
social skills, building on previously established gains, 
meeting age-specific developmental needs, maximizing 
youth engagement, or providing par¬ 
ticipants with variety 

A third domain addressed 
across the afterschool program 
quality frameworks is strong part¬ 
nerships with families, schools, and 
other community organizations. 

These partnerships are considered 
important for several reasons. Part¬ 
nerships with families can facilitate 
youth engagement in the program. 

Partnerships with schools enable 
afterschool programs to coordinate 
the content of their services with 
school-based learning. Linkages to 
other community organizations can 
assist afterschool programs by mak¬ 
ing their services widely accessible to young people and 
by using the afterschool program to reinforce skills 
learned in other settings. 

Another domain consistently cited in the various 
quality frameworks is the importance of young people’s 
active participation and engagement in program activi¬ 
ties. Several of the quality frameworks emphasize youth 
engagement, citing evidence that positive outcomes are 
more likely when youth participate in a program more 
frequently and over a longer period of time. However, it 
appears that the links between program engagement and 
positive youth outcomes are complex, multi-faceted, 
and holistic rather than direct. For instance, youth par¬ 
ticipation and engagement may be a predictor of posi¬ 
tive youth outcomes, but they may also be an outcome 
of a quality program. Various formulations of program 
quality highlight programmatic factors such as availabil¬ 
ity of programming—how long the program is open, 
whether it is located where potential participants can 
access it—as important in affecting young people’s lev¬ 
els of participation. Several quality frameworks suggest 
that offering a broad range of program activities is 
important in fostering youth engagement. Formulations 
of program quality consistently relate youth engagement 
to quality in the areas of supportive relationships, inten¬ 
tional programming, and outside partnerships. 

A fifth common theme across the quality frame¬ 
works is the importance of a healthy, physically safe 
environment. Some of the quality frameworks reference 


the availability of nutritious snacks and opportunities 
for physical activity as important dimensions of quality. 
Having adequate space, supervision, and physical secu¬ 
rity is consistently associated with program quality. In 
some cases, physical safety and psy¬ 
chological safety are considered to 
be interrelated aspects of quality. 
While the literature on program 
quality does not describe in detail 
the indicators of physical safety, 
definitions of program quality seem 
to take physical safety as an essen¬ 
tial, foundational dimension of pro¬ 
gram quality. 

A final point of convergence 
across the definitions of program 
quality is the domain of program 
management, particularly manage¬ 
ment practices that support pro¬ 
gram sustainability and continuous 
program improvement. Several of 
the quality frameworks associate quality with having 
evaluation practices in place, with engaging in frequent 
assessment of program practices, and with using assess¬ 
ment to improve the program. Additionally, several of 
the frameworks point to the importance of high-quality 
program management and self-evaluation in promoting 
staff development and program activities. This correla¬ 
tion suggests that continuous program improvement, as 
one dimension of program quality, supports quality in 
other key domains of program functioning. 

These six domains—supportive relationships, inten¬ 
tional programming, strong community partnerships, 
promotion of youth engagement, physical safety, and 
continuous quality improvement—represent clear points 
of convergence across the various definitions of program 
quality. The field is reaching a consensus regarding what 
aspects of program quality are important and how these 
dimensions of program quality fit into the overall picture 
of afterschool programming. See Figure 1 on page 10. 

The Big Picture: Putting Quality 
Frameworks into Action 

Considered collectively, the literature reviewed in this arti¬ 
cle indicates that the afterschool field is reaching agree¬ 
ment on several key points related to program quality. 

Afterschool programs can be effective in enhancing 
academic achievement and social and emotional devel¬ 
opment. Programs that have a positive effect in one of 
these domains tend to achieve positive benefits in the 


Afterschool programs can 
be effective in enhancing 
academic achievement 
and social and emotional 
development. Programs 
that have a positive 
effect in one of these 
domains tend to achieve 
positive benefits in the 
others as well. 
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others as well. That is, successful programs appear to 
affect a range of outcomes, including academic perfor¬ 
mance and social and emotional development. 

ft also is clear that not all afterschool programs are 
equally effective. Experts and researchers have reached 
a general consensus that successful afterschool programs 
have a number of characteristics in common. The liter¬ 
ature now offers several formulations of program qual¬ 
ity that are rooted in the results of individual outcome 
studies, meta-analyses of outcome research, and a grow¬ 
ing body of process research. At the same time, after¬ 
school professionals have access to a variety of quality 


assessment tools that reflect the various ways that pro¬ 
fessionals “in the field” are defining program quality. 
Across the literature, formulations of program quality 
converge on common ground 
regarding what constitutes after¬ 
school program quality. Our review 
suggests that six domains—sup¬ 
portive relationships, intentional 
programming, strong community 
partnerships, promotion of youth 
engagement, physical safety, and 
continuous quality improvement— 
represent the field’s consensus on 
program quality. 

Having established a relatively 
consistent set of quality indicators allows us to engage in 
activities that are useful to all stakeholders in afterschool 
programming, including: 


• Assessing the degree to which a program is likely to 
produce positive youth outcomes 

• Providing guidelines for implementing program 
improvements 

• Developing guidelines for staff training 

• Establishing a baseline for assessing changes in a pro¬ 
grams quality over time 

As the field gathers additional empirical data 
regarding what program practices define quality, it will 
also need to scientifically validate the measures it uses 
to assess quality, a process that has already begun 

(Yohalem & Wilson- 
Ahlstrom, 2009). 
Moreover, as the after¬ 
school field moves 
forward with imple¬ 
menting quality stan¬ 
dards, a number of 
issues must be kept in 
mind. For example, 
although the field 
is paying increasing 
attention—both theo¬ 
retical and empiri¬ 
cal—to afterschool 
program quality, there 
remains relatively lit¬ 
tle research on how 
to implement quality 
standards and whether 
doing so actually 
improves programs (Granger et al., 2007). Granger and 
colleagues have emphasized the importance of con¬ 
ducting research on how to best implement quality 
improvement processes. 

Additionally, from a practical 
standpoint, quality improvement 
requires working effectively with 
limited resources and prioritizing 
the dimensions of quality to be tar¬ 
geted. The frameworks of Metz 
and colleagues (2008) and of 
Pittman and colleagues (2008) 
offer suggestions for prioritizing 
various dimensions of quality. 
Pittman and colleagues emphasize 
safety and supportive relationships as the foundation 
that must be established before other dimensions of 
quality can be achieved. Metz and colleagues suggest 


Figure 1: Unified Picture of Afterschool Program Quality 
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Focused and intentional 
programming, engaging 
activities, and supportive 
staff relationships may be 
necessary, but they are 
not sufficient to ensure 
program success. 
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that focused, intentional activities and continuous pro¬ 
gram improvement are the essential features for achiev¬ 
ing overall program quality. 

In addition to these practical considerations, the 
afterschool field faces the broader theoretical and empir¬ 
ical task of determining how high-quality program prac¬ 
tices affect, and are affected by, other factors known to 
be essential to positive youth outcomes. Research sug¬ 
gests that a high-quality program does not operate in 
isolation (Vandell, Reisner, & Peirce, 2007). Focused 
and intentional programming, engaging activities, and 
supportive staff relationships may be necessary, but they 
are not sufficient to ensure program success. Rather, an 
effective program requires successful interactions with 
the community (Smith & Van Egeren, 2008) and con¬ 
tinual efforts to recruit and retain actively engaged youth 
who then become committed to the program (Shernoff 
& Vandell, 2008). As the afterschool field continues to 
refine its definition of program quality, quality standards 
ought to encourage practices that not only are associated 
with positive outcomes but also make the most of these 
moderating factors. 

The field is becoming increasingly focused on the 
broader context in which afterschool programs oper¬ 
ate. By utilizing both outcome and process research in 
a quality-driven model, the afterschool field is poised 
to undertake important program improvement efforts 
that result in broad, holistic benefits for the young peo¬ 
ple we serve. 
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